This Page is Out of Date Some of the information on this page is no longer applicable. Please see our three-part tutorial on compiling and linking code on HPC instead: HPC Compilation and Job Submission Tutorial.

Compiling serial code on hpc

We have two complete sets of compilers available on hpc, one from the GNU Project and one from the Portland Group, Inc. (PGI). The GNU compilers gcc and g77 are for C/C++ and Fortran 77 code, respectively. The PGI compilers pgcc, pgCC, pgf77, and pgf95 are for C, C++, Fortran 77, and Fortran 95 code, respectively. All compilers have extensive man pages listing their many options.

To check serial compiling, consider the most basic, ANSI-C compliant, serial Hello-world program hello.c in the C language:

#include <stdio.h>

int main (int argc, char *argv[])
{
  printf ("Hello, world!\n");

  return 0;
}

To compile it with gcc and name the output file hello_gcc, say

$ gcc hello.c -o hello_gcc

Analogously, to test the PGI compiler, say

$ pgcc hello.c -o hello_pgcc

You can execute them by saying ./hello_gcc and ./hello_pgcc and should get a "Hello, world!" in response. Notice that you are only allowed to run code on the user node for such brief tests.

Compiling and executing C++ or Fortran code is analogous.

Available versions of MPI

To compile parallel code using MPI (Message Passing Interface) commands, it is best to use the wrapper compilers mpicc, mpic++/mpicxx/mpiCC, mpif77, and mpif90 for C, C++, Fortran 77, and Fortran 90 code (instead of calling one of the serial compilers directly and linking to the MPI libraries).

You will notice that the previous statement did not specify the actual compiler family used, GNU or PGI. So, how does, e.g., mpicc know which C compiler to use? To specify the programming environment completely, you still have to select which compiler to use.

But moreover, there are actually several implementations of MPI available on hpc, see below how to list them. All implementations are available in combination with both GNU and PGI compiler suites. The current default on hpc is the implementation from Open MPI together with the PGI compiler suite.

As a matter of background, I mention that for successful compiling, linking, and running, several things have to come together correctly: Your path has to include the location of the correct compile script, the MPI header files have to be found, and the correct MPI libraries have to be found. This is all conveniently set up correctly for you by the switcher utility. See its man page for more information (man switcher), but the most needed commands are these:

• You can list all available MPI implementations by "switcher mpi --list", which lists

  $ switcher mpi --list
  gcc-openmpi-1.2.5
  openmpi-1.2.4
  mpich-ch_p4-gcc-1.2.7
  pgi-mvapich2-1.0.2
  gcc-mvapich-1.0.0
  pgi-openmpi-1.2.5
  lam-7.1.4
  pgi-mvapich-1.0.0
  gcc-mvapich2-1.0.2
  

  The names involving pgi and gcc indicate the compilation against the PGI and GNU compilers, respectively. (I have not tested the other combinations of MPI implementations and compilers myself; if someone does test any, please let me know!)

• You can confirm the MPI implementation that you are currently set up for by saying "switcher mpi --show". Unless you switched the implementation, this should show

  $ switcher mpi --show
  system:default=pgi-openmpi-1.2.5
  user:exists=1
  

  Notice the "pgi" and "openmpi" in the name. You can further confirm with "which mpicc" that the wrapper compiler mpicc is served from the directory /usr/mpi/pgi/openmpi-1.2.5/bin/. If you are satisfied with this default choice, you do not need to do anything else.

• If you want to switch to another combination of MPI implementation and compiler, for instance, to use the GNU compiler suite with Open MPI, you would say "switcher mpi = gcc-openmpi-1.2.5", where the possible implementations are the ones listed above by "switcher mpi --list".
  The "switcher mpi --show" can now confirm your choice. Notice that this selection remains in effect across sessions, so you will only need to switch once. But notice carefully that this selection becomes "effective for future shells", as it correct states; so, you either have to log out and log in again before attempting to compile and run MPI code.

Compiling parallel code with MPI on hpc

Let's assume that you have the (nearly) simplest parallel generalization of the serial "Hello, world!" program given by the program hello_parallel.c:

#include <stdio.h>
#include "mpi.h"

int main (int argc, char *argv[])
{
  int id, np;
  char name[MPI_MAX_PROCESSOR_NAME];
  int namelen;

  MPI_Init (&argc, &argv);

  MPI_Comm_size (MPI_COMM_WORLD, &np);
  MPI_Comm_rank (MPI_COMM_WORLD, &id);
  MPI_Get_processor_name (name, &namelen);

  printf ("This is Process %3d out of %3d running on host %s\n", id, np, name);

  MPI_Finalize ();

  return 0;
}

Clearly, the simplest generalization of hello.c above would be to just say "Hello, world!" from all processes. For that, you would only need the line that includes mpi.h, the MPI_Init call, and the MPI_Finalize call. The printf would then execute on each parallel process. To make the code a little more useful and demonstrate MPI commands that will appear in practically every MPI code, the MPI_Comm_size and MPI_Comm_rank obtain the number of all parallel processes of the code (its size) and the ID number (the rank) of each process, respectively. Additionally, MPI_Get_processor_name is a nice MPI command that obtains the hostname of the node on which the process runs.

The compilation and linking of your code should work just like compiling in serial, except the compiler is replaced by the wrapper compiler mpicc. Hence, compile and link the code both in one step by

mpicc hello_parallel.c -o hello_parallel

The script mpicc works on the surface just like any regular compiler. For instance, the option "-o hello_parallel" chooses the name of the output file (here the executable file), and mpicc compiles and links in (seemingly) one step.

More formally, I could also separate the compile from the link step. That is, the C file hello_parallel.c is first compiled into object code hello_parallel.o (default output name, if the option -c is used), which then gets linked to the required libraries to obtain the executable hello_parallel. The sequence of the two commands

mpicc -c hello_parallel.c
mpicc hello_parallel.o -o hello_parallel

accomplishes this, where the option -c stands for "compile-only".

See the man page of mpicc for more information. If you want to compile C++ or Fortran programs, the logic of the commands is the same as above. Simply replace mpicc by mpic++/mpicxx/mpiCC, mpif77, or mpif90. See their man pages for additional information.

Notice again that the MPI wrapper compilers mpicc, mpic++/mpicxx/mpiCC, mpif77, and mpif90 have the same name for each of the MPI implementations available. Which one is used depends on which implementation you have selected with switcher. Notice also from their man pages that any option that is not recognized as a MPI option is passed through to the underlying compiler; so you might want to know the name of the underlying compiler to be able to pull up its man page. To this end, it might be useful to make doubly sure which compiler is accessed by your MPI compile script, you can use the "-show" option as in

mpicc hello_parallel.c -o hello_parallel -show

This option causes mpicc to show what compile command it would issue without actually issuing them; then notice at the beginning of the line what compiler you see.

hello.c (text/x-csrc)